Field of the Invention
The present invention is a method and system for rating in-store media elements based on the measurement for behavior patterns and demographics of the audience in the vicinity of the media element, where a plurality of input images of the audience are captured by at least a means for capturing images in the vicinity of the media element and the input images are processed by automated video analytic algorithms in order to measure the behavior patterns and demographics of each person in the audience tied to the media element.
Background of the Invention
U.S. Pat. No. 7,006,982 of Sorensen (hereinafter Sorensen) disclosed a system for analyzing purchase selection behavior utilizing a visibility measure. The system uses mathematical models to define the line of sight and viewing area of the shoppers, and then estimates the products the shoppers may have seen based on their shopper path. The key difference between the system disclosed by Sorensen and the current invention is that Sorensen's system only measures the likelihood that a passing-by shopper may view the product. The framework disclosed in the present invention makes it possible for us to provide a much more scalable shopper behavior measurement system. The current invention can measure the number of shoppers passing by a media element, their dwell time, age, gender, and ethnicity, not possible using Sorensen's system. Sorensen's system also focuses on estimating exposure to products on the shelves, whereas the current invention measures and estimates the exposure and engagement received by media elements. The goal of the two measurement systems is also different—Sorensen's system measures the likelihood that a product is viewed and hence purchased by the shopper, whereas the current invention develops detailed media metrics that will help retailers and advertisers buy and sell in-store media as a measured media. Sorensen also does not use automated vision algorithms for collecting shopper data. The use of vision algorithms enables automated collection of unique data not possible with any other technology.
U.S. Pat. Appl. Pub. No. 20030039379 of Gutta, et al. (hereinafter Gutta) disclosed a system for automatically assessing interest in a displayed product. The method includes capturing image data within a predetermined proximity of the displayed product, identifying people in the captured image data, and assessing the interest in the displayed product based upon the identified people. The key difference between the system disclosed by Gutta and the current invention is that Gutta does not formulate a scalable framework for collecting and analyzing data over large periods of time. The framework disclosed in the present invention makes it possible for us to analyze the shopper exposure to and engagement with different media elements over time and develop media ratios.
U.S. Pat. Appl. Pub. No. 20020161651 of Godsey, et al. (hereinafter Godsey) disclosed a system for tracking consumers in a store environment. The system tracks a plurality of product containers in a store environment and generates a track through the store environment representative of a continuous path followed by each of the product containers to a point-of-sale location. The system disclosed by Godsey is different from the current invention in many ways. Unlike the current invention, Godsey's system does not track shoppers in the stores; it tracks product containers and incorrectly assumes that the product containers transverse the store with the shoppers. This assumption leads to inaccurate data and, consequently, wrong interpretations. Another disadvantage of the system is that it cannot collect the unique types of data, such as engagement and demographics, as can be done by the current invention.
U.S. Pat. No. 7,006,979 of Samra, et al. (hereinafter Samra U.S. Pat. No. 7,006,979) and U.S. Pat. No. 7,003,476 of Samra, et al. (hereinafter Samra U.S. Pat. No. 7,003,476) disclosed systems for analyzing the success of a marketing campaign and for defining targeted marketing campaigns using embedded models and historical data. The systems are dissimilar from the current invention because they are not focused on impact measurement of in-store media elements and are not based on unique vision algorithms.
U.S. Pat. No. 6,286,005 of Cannon (hereinafter Cannon) disclosed computer-based systems for analyzing audience data. Cannon discloses a method and apparatus for quickly and easily retrieving, manipulating, and analyzing large quantities of computer-based data relevant to television-viewing consumers. The key differences between the systems disclosed by Cannon and the one in the current invention are that 1) Cannon primarily focused on a television audience, 2) the process used for data collection is different, 3) measurements provided by two systems are different.
First, the definition of audience is very different from television and in-store media elements. For television measurement, audience is defined as a household which had the television turned on when content was played, whereas for in-store media elements, audience is defined as the group of persons who pass by the element. The definition of audience creates unique challenges in data collection and analysis; it also changes the types of analysis needed by advertisers. Cannon does not cover the challenges offered by digital signage.
Secondly, the system described by Cannon processes data primarily collected through exit interviews, telephone interviews, online surveys, etc., which require active participation from the audience members, whereas the current invention uses automated video-based data collection to acquire the data. The types of data collected are very different. For example, Cannon discusses that the demographic information can include information such as a viewer's age, geographical location, income, and level of education, which cannot be calculated using computer vision algorithms. Therefore, the definition of the demographic information in Cannon is different from that of the current invention. Applicant's demographic information is primarily concerned with the audience in the vicinity of a digital signage, whereas the demographic information in Cannon is primarily concerned with the television-viewing consumers, so the approaches as to how the demographic information is gathered and applied in the embodiments are significantly different between Cannon and the current invention.
Thirdly, the current invention provides unique measurements not offered by Cannon. The current invention includes analyses of the actual time audience members spend in the vicinity of the element, the emotional impact of the content on the audience, and demographic segmentation based on automated estimation of age, gender, ethnicity, and shopping behavior. None of these parameters are measured or analyzed by the system proposed by Cannon.
U.S. Pat. No. 6,516,464, U.S. Pat. No. 6,228,038, and U.S. Pat. No. 6,045,226 of Claessens (hereinafter Claessens) disclosed a system for detecting audience response to audio visual stimuli. The system disclosed in the patent requires a panel of viewers to watch the content and use a computer-based system to respond to it in real time. The viewer can register his or her likes or dislikes and qualitatively explain the response. The data is then used to evaluate the content. The key differences between Claessens' system and the current invention are that Claessens' system requires active participation from the audience and can measure the effectiveness of a piece of content but not of a whole content. The current invention is superior because it directly measures the viewer behavior, is more accurate, and does not require active participation from viewers.
In U.S. Pat. No. 6,045,226, Claessens disclosed a system for measuring the visual attention of subjects for a visible object. The disclosed system emits an invisible beam of light to scan the area of interest. The beam of light is reflected from the retina and/or cornea of said person(s) or animal(s). This reflected beam is used to estimate the direction in which the subject is looking and the duration of view. The system can be used to measure the duration for which a person looks directly at a signage. The key differences between Claessens' system and the current invention are that Claessens' system uses a specified source of radiation to measure the duration of view, and it cannot provide all of the data provided by the current system, such as segmentation and shopping behavior.
U.S. Pat. No. 7,302,475 of Gold, et al. (hereinafter Gold) disclosed a system for measuring reactions to product packaging, advertising, or product features over a computer-based network. The system depends on a web platform to present different images to the respondents and collect their responses using online surveys. The current invention is different from the system disclosed by Gold because it does not depend on audience involvement, and the data is collected unobtrusively.
U.S. Pat. No. 5,991,734 of Moulson (hereinafter Moulson) disclosed a system for measuring the creative value in communications. The system relies on proactive participation from the respondents to collect feedback on creativity of the media. The current invention is different from Moulson's disclosed system because it does not depend on audience involvement, and the data is collected unobtrusively.
U.S. Pat. No. 7,374,096 of Overhultz, et al. (hereinafter Overhultz) disclosed a system for advertising compliance monitoring. The system uses RFID signals to detect the presence and absence of the audience members and of the marketing stimuli. Overhultz' system is different from the current invention because it focuses on measuring whether or not the media element is in the correct location and in the correct orientation, whereas the current invention measures the audience traffic and engagement details. The technologies used by the two patents are also different.
U.S. Pat. No. 6,563,423 of Smith (hereinafter Smith) disclosed a location tracking system to track the movements of customers. This tracking is used to determine a customer's pace, how long a customer stayed at a particular location, and to determine how many customers passed a location. The purchase of a customer is determined by reading the tagged code at the time of purchase and relating this to whether or not the customer visited a particular display earlier.
The system disclosed by Smith is different from the current system because it uses magnetic tags to track customers in the retail space which cannot collect all forms of data collected by the current system, and it does not disclose systems and processes to convert this data to media ratings.